Temporal Factorization

The traditional subspace-based approaches to segmentation (often referred to as multi-body factorization approaches) provide spatial clustering/segmentation by grouping together points moving with consistent motions. We are exploring a dual approach to factorization, i.e., obtaining temporal clustering/segmentation by grouping together frames capturing consistent shapes. Temporal cuts are thus detected at non-rigid changes in the shape of the scene/object. In addition it provides a clustering of frames with consistent shape (but not necessarily same motion). For example, in a sequence showing a face which appears serious at some frames, and is smiling in other frames, all the “serious expression” frames will be grouped together and separated from all the “smile” frames which will be classified as a second group, even though the head may meanwhile undergo various random motions.